Systems and methods for categorizing network traffic content

ABSTRACT

A method for categorizing network traffic content includes determining a first characterization of the network traffic content determining a first probability of accuracy associated with the first characterization, and categorizing the network traffic content based at least in part on the first characterization and the first probability of accuracy. A method for use in a process to categorize network traffic content includes obtaining a plurality of data, each of the plurality of data representing a probability of accuracy of a characterization of network traffic content, and associating each of the plurality of data with a technique for characterizing network traffic content. A method for categorizing network traffic content includes determining a characterization of the network traffic content, determining a weight value associated with the characterization, and categorizing network traffic content based at least in part on the characterization of the network traffic content and the weight value.

RELATED APPLICATION DATA

This application is a Continuation of U.S. application Ser. No.14/568,368, filed on Dec. 12, 2014; which is a Continuation of U.S.application Ser. No. 14/284,935, filed on May 22, 2014, issued on Jan.12, 2016 as U.S. Pat. No. 9,237,160; which is a Continuation of U.S.application Ser. No. 13/795,390, filed on Mar. 12, 2013, issued on Jul.15, 2014 as U.S. Pat. No. 8,782,223; which is a Continuation of U.S.application Ser. No. 13/153,889, filed on Jun. 6, 2011, issued on Jan.21, 2014 as U.S. Pat. No. 8,635,336; which is a Continuation of U.S.application Ser. No. 12/403,996, filed on Mar. 13, 2009, issued on Jul.12, 2011 as U.S. Pat. No. 7,979,543; which is a Continuation of U.S.application Ser. No. 10/993,629, filed on Nov. 19, 2004, issued on Jul.21, 2009 as U.S. Pat. No. 7,565,445; which claims the benefit ofpriority to U.S. Provisional Application No. 60/580,841, filed Jun. 18,2004, to each of which priority is claimed and the entirety of each ofwhich is incorporated herein by reference.

BACKGROUND

The field of the invention relates to computer systems and computernetworks, and more particularly, to systems and methods for categorizingcontent of computer and network traffic.

Many organizations face the challenge of dealing with inappropriatecontent, such as email spam, misuse of networks in the form of browsingor downloading inappropriate content, and use of the network fornon-productive tasks. Many organizations are struggling to controlaccess to appropriate content without unduly restricting access tolegitimate material and services. Currently, a common solution forblocking unwanted Web activity is to block access to a list of banned orblacklisted websites and pages based on their URLs. However, suchapproach may be unnecessarily restrictive, preventing access to validcontent in websites that may contain only a limited amount ofundesirable material. Also, the list of blocked URLs requires constantupdating.

Many email spam elimination systems also use blacklists to eliminateunwanted email messages. These systems match incoming email messagesagainst a list of mail servers that have been pre-identified to be spamhosts, and prevent user access of messages from these servers. However,spammers often launch email spam from different hosts every time, makingit difficult to maintain a list of spam servers.

It would be desirable to categorize network traffic content, and preventundesirable network traffic content (e.g., content that belongs to anundesirable category) to be passed to users. Currently, many contentdetecting systems use human based categorization to categorize networkcontent. In such systems, an operator manually analyzes network content,then uses the results of the analysis to categorize the network content.Although such techniques may produce reliable results, they are laborintensive and time consuming.

In another technique, HTML links are analyzed to determine acharacteristic of network content. However, such technique mayerroneously mischaracterize network content. Companies have also usedother techniques for characterizing network content, but each of thesetechniques may not produce reliable result.

Accordingly, new systems and methods for categorizing content ofcomputer and network traffic would be useful.

SUMMARY

In accordance with some embodiments, a method for categorizing networktraffic content includes determining a first characterization of thenetwork traffic content, determining a first probability of accuracyassociated with the first characterization, and categorizing the networktraffic content based at least in part on the first characterization andthe first probability of accuracy.

In accordance with other embodiments, a computer product includes acomputer-readable medium, the computer-readable medium having a set ofstored instructions, an execution of which causes a process to beperformed, the process comprising determining a first characterizationof the network traffic content, determining a first probability ofaccuracy associated with the first characterization, and categorizingthe network traffic content based at least in part on the firstcharacterization and the first probability of accuracy.

In accordance with other embodiments, a device for categorizing networktraffic content includes means for determining a first characterizationof the network traffic content, means for determining a firstprobability of accuracy associated with the first characterization, andmeans for categorizing the network traffic content based at least inpart on the first characterization and the first probability ofaccuracy.

In accordance with other embodiments, a method for use in a process tocategorize network traffic content includes obtaining a plurality ofdata, each of the plurality of data representing a probability ofaccuracy of a characterization of network traffic content, andassociating each of the plurality of data with a technique forcharacterizing network traffic content.

In accordance with other embodiments, a computer product includes acomputer-readable medium, the computer-readable medium having a set ofstored instructions, an execution of which causes a process to beperformed, the process comprising obtaining a plurality of data, each ofthe plurality of data representing a probability of accuracy of acharacterization of network traffic content, and associating each of theplurality of data with a technique for characterizing network trafficcontent.

In accordance with other embodiments, a device for categorizing networktraffic content includes means for obtaining a plurality of data, eachof the plurality of data representing a probability of accuracy of acharacterization of network traffic content, and means for associatingeach of the plurality of data with a technique for characterizingnetwork traffic content.

In accordance with other embodiments, a method for categorizing networktraffic content includes determining a characterization of the networktraffic content, determining a weight value associated with thecharacterization, and categorizing network traffic content based atleast in part on the characterization of the network traffic content andthe weight value.

In accordance with other embodiments, a computer product includes acomputer-readable medium, the computer-readable medium having a set ofstored instructions, an execution of which causes a process to beperformed, the process comprising determining a characterization of thenetwork traffic content, determining a weight value associated with thecharacterization, and categorizing network traffic content based atleast in part on the characterization of the network traffic content andthe weight value.

In accordance with other embodiments, a device for categorizing networktraffic content includes means for determining a characterization of thenetwork traffic content, means for determining a weight value associatedwith the characterization, and means for categorizing network trafficcontent based at least in part on the characterization of the networktraffic content and the weight value.

In accordance with other embodiments, a method for categorizing networktraffic content includes determining a first characterization of thenetwork traffic content, determining a first probability of accuracyassociated with the first characterization, determining a secondcharacterization of the network traffic content using a secondtechnique, wherein the second technique is different from the firsttechnique, determining a second probability of accuracy associated withthe second characterization, and categorizing the network trafficcontent based at least in part on the first characterization, the secondcharacterization, the first probability of accuracy, and the secondprobability of accuracy.

In accordance with other embodiments, a system for categorizing networktraffic content includes means for determining a first characterizationof the network traffic content, means for determining a firstprobability of accuracy associated with the first characterization,means for determining a second characterization of the network trafficcontent using a second technique, wherein the second technique isdifferent from the first technique, means for determining a secondprobability of accuracy associated with the second characterization, andmeans for categorizing the network traffic content based at least inpart on the first characterization, the second characterization, thefirst probability of accuracy, and the second probability of accuracy.

In accordance with other embodiments, a computer product includes acomputer-readable medium, the computer-readable medium having a set ofstored instructions, an execution of which causes a process to beperformed, the process comprising determining a first characterizationof the network traffic content, determining a first probability ofaccuracy associated with the first characterization, determining asecond characterization of the network traffic content using a secondtechnique, wherein the second technique is different from the firsttechnique, determining a second probability of accuracy associated withthe second characterization, and categorizing the network trafficcontent based at least in part on the first characterization, the secondcharacterization, the first probability of accuracy, and the secondprobability of accuracy.

Other aspects and features will be evident from reading the followingdetailed description of the preferred embodiments, which are intended toillustrate, not limit, the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings illustrate the design and utility of various embodiments,in which similar elements are referred to by common reference numerals.More particular descriptions will be rendered by reference to specificembodiments, which are illustrated in the accompanying drawings.Understanding that these drawings are not to be considered limiting inscope, the embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying figures.

FIG. 1 illustrates a block diagram representing a system that includes amodule for categorizing network traffic content in accordance with someembodiments;

FIG. 2 illustrates a method for categorizing network traffic content inaccordance with some embodiments;

FIG. 3 illustrates ˜n example of results obtained using differenttechniques for characterizing network traffic content in accordance withsome embodiments;

FIG. 4 illustrates a method for categorizing network traffic contentusing an accuracy of a technique result in accordance with someembodiments;

FIG. 5 illustrates a method for categorizing network traffic contentusing an accuracy of a technique result in accordance with otherembodiments;

FIG. 6 illustrates a method for categorizing network traffic contentusing an accuracy of a technique result in accordance with otherembodiments; and

FIG. 7 is a diagram of a computer hardware system with which embodimentsof the present invention can be implemented.

DETAILED DESCRIPTION

Various embodiments are described hereinafter with reference to thefigures. It should be noted that the figures are not drawn to scale andthat elements of similar structures or functions are represented by likereference numerals throughout the figures. It should also be noted thatthe figures are only intended to facilitate the description of specificembodiments, and are not intended as an exhaustive description of theinvention, or as a limitation on the scope of the invention. Inaddition, an illustrated embodiment need not have all the aspects oradvantages of the invention shown. An aspect or an advantage describedin conjunction with a particular embodiment is not necessarily limitedto that embodiment and can be practiced in any other embodiments even ifnot so illustrated.

FIG. 1 illustrates a block diagram of a system 100, which includes acontent categorization module 110 for categorizing network trafficcontent in accordance with some embodiments. Sender 102 transmits dataassociated with network traffic content, such as email content or Webcontent, to module 110. Module 110 receives the transmitted data,determines a category to which the network traffic content belongs(e.g., categorize the network traffic content), and causes a result,such as a message, to be sent to a user 104. The message sent to user104 notifies the user 104 that a content belonging to a category hasbeen detected. As used in this specification, the term “user” should notbe limited to a human user, and can include a server or other types ofdevices that can receive information. Also, as used in thisspecification, the term “sender” should not be limited to a humansender, and can include a server or other types of devices that cantransmit information.

In some embodiments, module 110 can be implemented using software. Forexample, module 110 can be implemented using software that is loadedonto a user's computer, a server, or other types of memory, such as adisk or a CD-ROM. In some cases, module 110 can be implemented as Webapplications. In alternative embodiments, module 110 can be implementedusing hardware. For example, in some embodiments, module 110 includes anapplication-specific integrated circuit (ASIC), such as a semi-customASIC processor or a programmable ASIC processor. ASICs, such as thosedescribed in Application-Specific Integrated Circuits by Michael J. S.Smith, Addison-Wesley Pub Co. (1st Edition, June 1997), are well knownin the art of circuit design, and therefore will not be described infurther detail herein. In other embodiments, module 110 can also be anyof a variety of circuits or devices that are capable of performing thefunctions described herein. For example, in alternative embodiments,module 110 can include a general purpose processor, such as a Pentiumprocessor. In other embodiments, module 110 can be implemented using acombination of software and hardware. In some embodiments, module 110may be implemented as a firewall, a component of a firewall, or acomponent that is configured to be coupled to a firewall.

FIG. 2 illustrates a method 200 for categorizing network traffic contentin accordance with some embodiments. First, module 110 receives dataassociated with network traffic content and analyzes the content usingone or more techniques (Step 202). The techniques used will depend onthe type of content being analyzed. For the purpose of the followingdiscussion, it is assumed that the content being analyzed is Web-relatedcontent. However, it should be understood by those skilled in the artthat the process 200 of FIG. 2 can be used to categorize other types ofcontent.

The illustrated embodiments use four representative techniques (whichmay be tests) to independently analyze and characterize the Web-relatedcontent. The first representative technique involves performing anInternal Link Analysis (ILA). ILA is based on the theory that a websitemay have HTML links to other sites with similar characteristics. In suchcases, module 110 includes an ILA engine for analyzing HTML links withinan URL (with predefined depth—number of layers of sub-page). Using suchtechnique, if the majority of links associated with the Web contentpoints to a URL associated with a certain characterization (e.g.,educational, advertising, etc.), the Web content is determined(characterized) as having such characterization. The number of availablecharacterizations associated with the ILA technique can vary. Forexample, in some embodiments, four possible characterizations (referencecharacterizations) can be made available: educational, advertising,porno, and news.

The second representative technique involves performing an External LinkAnalysis (ELA). ELA is based on the theory that if most people considera website as having a certain characterization (e.g., educational,advertising, etc.), the website is then determined to have suchcharacterization. In such cases, a database is maintained (e.g., withinmodule 10 or in a separate memory) that points to a number of prescribedexternal portal sites that have some categorization of URLs. The portallist is constantly updated and maintained. The portal list can be storedin module 10 or in a separate memory that is coupled to module 10.

The third representative technique involves performing a Meta-TagAnalysis (MTA). MTA is based on the theory that if a majority or asubstantial number of meta-tags are associated with a certaincharacterization (e.g., educational, advertising, etc.), the website isdetermined as having such characterization. In such cases, module 110analyzes HTML content and determines if it contains any meta-tagsindicating which characterization the content belongs to. For example,if a majority or a substantial number of meta-tags are associated with acertain characterization, module 110 then determines the content ashaving such characterization.

The fourth representative technique involves performing a Token Analysis(TKA). TKA is based on the theory that if content contains a largenumber of words having the same attribute, then the content ischaracterized as having the attribute. In such cases, a database ofwords and phrases is maintained (e.g., in module 110 or in a separatememory). The words and phrases are abstracted to tokens, which can beused by module 110 to analyze a HTML file's content and determine acharacterization for the content. For example, if the content beinganalyzed contains a plurality of texts, the content can first benormalized to make it more “standard” for processing. Normalization ofcontent, can be performed by, for example, converting all letters toupper case, and replacement of all white space characters andpunctuation marks with one character (such as a single white space).Next, the database of words may be accessed. In such a database, wordsare represented by numbers, and each number is associated with one ormore attribute. Since every word is represented by a number, module 110can perform counting, and determines the frequency that these wordsappear in the content. If the content contains a large number of wordshaving the same attribute (e.g., educational, pornographic, etc.), thenthe content is determined as having the attribute (characterization). Itshould be noted that instead of using TKA to analyze text, in otherembodiments, TKA can be implemented to analyze graphic(s), orcombination of text and graphic(s).

Although four representative techniques for characterizing networktraffic content have been described, the scope of the invention shouldnot be so limited. In other embodiments, instead of using fourtechniques, module 110 can use a different number of techniques toanalyze network traffic content. Also, in other embodiments, instead ofthe four techniques described previously, module 110 can employdifferent techniques (e.g., variations of the previously describedtechniques) or different combination of techniques to analyze networktraffic content of various types.

After network traffic content has been analyzed and characterized,module 110 determines a category to which the content belongs (i.e.,categorizes the content) based on the characterization by the fourrepresentative techniques (Step 204). In the illustrated embodiments,the category is determined by associating each technique with anaccuracy. The accuracy associated with each technique represents theaccuracy of the characterization determined by the technique.

FIG. 3 illustrates an example of results provided by the fourrepresentative techniques to characterize a website content. In theexample, ILA characterizes the website content as “Porno,” ELAcharacterizes the website content as “Educational,” MTA characterizesthe website content as “Porno,” and TKA characterizes the websitecontent as “News.” These characterizations are merely illustrative andcould be any of a number of other characterizations. Accuracies of thetechnique results for the respective four representative techniques arealso provided. In the illustrated example, the results provided by theILA, ELA, MTA, and TKA have a 10% accuracy (Le., a 10% probability thatthe result is accurate), a 20% accuracy, a 40% accuracy, and a 30%accuracy, respectively.

In some embodiments, module 110 is configured to determine theaccuracies associated with different techniques by receiving theaccuracies as inputs. For examples, a user can manually input theaccuracies into the module 110, or alternatively, the accuracies can betransmitted to the module 110 via a communication link. Alternatively,module 110 can be configured to calculate the accuracies.

Various methods can be employed to determine the accuracy (orprobability of accuracy). In the illustrated embodiments, the accuracycan be determined by performing statistical analysis for each of thefour techniques. For example, results provided by ILA can be comparedwith results provided by human categorization (i.e., categorizationperformed manually), and be analyzed to determine how accurate they are.Such analysis can be carried out for each of the available referencecharacterizations for each technique. For example, if ILA includes fourpossible reference characterizations (e.g., Porno, Educational, News,and Advertisement), then statistical analysis can be performed for eachof the four reference characterizations to determine how accurate of aresult the ILA can provide with respect to each referencecharacterization. If it is determined from the statistical analysis thatILA provides “Porno” characterization with 10% accuracy, “Educational”characterization with 30% accuracy, “News” characterization with 20%accuracy, and “Advertisement” characterization with 40% accuracy, thenevery time ILA characterizes network traffic content as “Porno,”“Educational,” “New,” or “Advertisement,” a 10%, 30%, 20%, or 40% willbe assigned to the result of characterization, respectively. As such,the accuracy (as expressed in probability value) can becharacterization-specific for each of the techniques used.

In other embodiments, accuracies for the categories associated with onetechnique (e.g., ILA) can be processed to determine an average accuracyfor the technique. In such cases, the same average accuracy associatedwith a technique is used for all possible characterizations for thetechnique. Using this technique for the above example, a 25%(=(10%+30%+20%+40%)/4) accuracy will be associated with results providedby the ILA, regardless of the characterization determined by the ILA.

Although accuracy of a technique has been described as a percentage ofprobability, in other embodiments, accuracy of a technique can berepresented by other variables. For example, in alternative embodiments,a weight value representing an accuracy of a result can be associatedwith each of the techniques used to analyze network traffic content. Insuch cases, the weight value can be determined based on probabilitystudies of results of the techniques, as similarly discussed previously.In some embodiments, a weight of 0 value can be used to associateresults of a technique when a probability study indicates that suchtechnique has an accuracy below a prescribed threshold (e.g., 10%), anda weight having a value of 2 can be used to associate result of atechnique when a probability study indicates that such technique has anaccuracy above a prescribed threshold (e.g., 50%).

Various techniques can be used to categorize network traffic contentbased on accuracies of the results provided.

FIG. 4 illustrates a technique for categorizing network traffic contentin accordance with some embodiments. As shown in FIG. 4, the step 204 ofcategorizing content includes obtaining technique result(s) from step202 (Step 402), and selecting the result that has the highest accuracy(Step 404). For example, module 110 can be configured or programmed tosearch for an accuracy that has the highest value, and select the resultassociated with the highest accuracy for categorizing network trafficcontent. Using such technique for the above example, module 110 willcategorize the content as “Porno” based on the result of the MTA becausethe MTA characterizes the content as “Porno” and has the highest degreeof accuracy (i.e., 40%).

FIG. 5 illustrates another technique for categorizing network trafficcontent in accordance with other embodiments. As shown in FIG. 5, thestep 204 of categorizing content includes obtaining technique result(s)from step 202 (Step 502), and combining accuracies of two or morecharacterizations if the characterizations yield the same (or similar)result (Step 504). Using such technique for the above example, module110 adds the probabilities of accuracy for the ILA and MTA (becausethese two analysis characterize the network traffic content as “Porno”),thereby producing a combined accuracy of 50% (=10%+40%). The combinedaccuracy for the characterization is then compared with other accuraciesor other combined accuracies for other characterizations, and thecharacterization associated with the maximum accuracy (or combinedaccuracy) is then selected to categorize the content (Step 506).Following the above example, the combined accuracy=50% for the “Porno”characterization is compared with the accuracies for the “News”characterization (=30%) and the “Educational” characterization (=10%).Because the “Porno” characterization has the maximum accuracy (=50%),module 110 categorizes the network traffic content as “Porno” based onthe combined results provided by the ILA and MTA.

In other embodiments, the module 110 is configured to determine whetheran accuracy associated with a result is below or above a prescribedthreshold (e.g., by performing a comparison between the accuracy and theprescribed threshold). In such cases, if the accuracy for a technique(e.g., ILA) is below the prescribed threshold, the module 110 thendisregards the result provided by the technique. On the other hand, ifthe accuracy for a technique is above the prescribed threshold, themodule 110 then accounts for the result provided by the technique in thecontent categorization process.

Other algorithms for categorizing network traffic content based onaccuracy of technique used can also be employed in other embodiments.For example, in alternative embodiments, the accuracy values can befurther processed (e.g., multiplied by one or more weight values), andthe processed values are then used to categorize network trafficcontent.

FIG. 6 illustrates a further technique for categorizing network trafficcontent in accordance with other embodiments, which does not involveperforming an algorithm to categorize network traffic content. As shownin FIG. 6, the step 204 of categorizing content includes obtainingtechnique result(s) from step 202 (Step 602), and presenting result(s)of characterization by one or more techniques (e.g., ILA) in conjunctionwith a variable that is associated with an accuracy or quality of theresult (Step 604) (FIG. 6). For example, in some embodiments, module 110categorizes network traffic content by providing a message to user 104,wherein the message includes information, such as those shown in FIG. 3,In such cases, module 110 does not summarily place the content in acategory, but allows user 104 to draw his/her conclusion based on theinformation presented.

It should be noted that although several methods for categorizationnetwork traffic content have been described with reference to accuraciesfor various techniques, the scope of the invention should not be solimited. In alternative embodiments, instead of or in addition toaccuracy, other variables can also be used to categorize network trafficcontent. For example, in other embodiments, a weight value or factor canbe associated with a result of a technique. The weight value canrepresent a characteristic of an analysis method, such as, usagefrequency (e.g., higher weight value can be assigned to an analysismethod that has a higher usage frequency, and vice versa), complexity ofanalysis technique (e.g., higher weight value can be assigned toanalysis technique that is more advanced or complex), or updatefrequency (e.g., higher weight value can be assigned to analysistechnique for which parameters for characterizing content are updatedrelatively more frequent). Also, in other embodiments, more than oneweight factors for each technique are used. In accordance with anotheraspect of the invention, a memory is provided for storing accuracyvalue(s) and/or weight value(s) for each of the technique(s). The memorycan be a disk, a computer hard drive, a server, or any device capable ofstoring electronic information. Such memory can be a component of module110, a component that is configured to integrate with module 110, or acomponent that is coupled to module 110 via a communication link (wireor wireless).

As illustrated by the above examples, each of the techniques contributesto a final decision of content categorization. By using more than onetechnique (e.g., ILA, ELA, MTA, TKA) to characterize network trafficcontent, user 104 can obtain a more accurate content categorization(because one analysis technique may be more accurate than another).Also, including accuracy and/or weight value(s) associated with atechnique result in a content categorization process is advantageousbecause they account for a quality and/or nature of the contentcategorization, thereby rendering the categorization result moreaccurate. In some cases, if the final decision is non-conclusive (e.g.,result does not meet prescribed criteria for categorizing content), thecontent will be flagged and be sent to an operator for manualverification.

Although embodiments of module 110 and process 200 have been describedwith reference to categorizing Web content, it should be understood thatthe same or similar methods and modules may also be used to categorizeother content, such as FTP, public server addresses, emails, telnetdata, instant messenger data, SSH data, SFTP data, and any electronicinformation. In some embodiments, module 110 includes a protocoldifferentiator (not shown), which examines headers of the networktraffic and determines the types of content being screened. Module 110then analyzes content using prescribed analysis method(s) (e.g., ILA)for the type of content. For example, if it is determined that thenetwork traffic content is Web content, then all four techniques (ILA,ELA, MTA, TKA) can be used to analyze the content. On the other hand, ifit is determined that the network traffic content is an email, then onlyTKA, for example, is used to analyze the content. In some embodiments,each type of network traffic content is pre-assigned to a port of anetwork gateway by a default configuration. For example, HTTP, SMTP,POP, IMAP, and FTP data may each be pre-assigned to be transmittedthrough a designated port. In such case, protocol differentiator candetermine a type of content based on an identification of a porttransmitting the network content. In other embodiments, protocoldifferentiator can be configured to scan all available ports in order todetermine a type of the network traffic content being screened.

Also, in other embodiments, instead of, or in addition to, determining acategorization of network traffic content, module 110 can be configuredto control flow of network traffic content based on a determinedcharacteristic of the network traffic content. For example, module 110can be configured to block HTTP request, thereby preventing undesirablecontent, such as a Web page containing undesirable content, from beingtransmitted to user 104. In other cases, module 110 may erase networktraffic content, or modify network traffic content based on a determinedcharacteristic of the network traffic content, such that only portion(s)of the content is allowed to pass to user 104.

In some embodiments, a user interface can be provided that allows user104 to select criteria or parameters for categorizing network trafficcontent. For example, module 110 can allow user 104 to set threshold(s)(e.g., minimum accuracy required before it should be considered), selectweight values to be considered in a content categorization process, anddesignate number of categories for each content (sometimes a content canbe categorized as belonging to more than one categories). Also, in otherembodiments, the module 110 can allow user 104 to select whichtechnique(s) (e.g., ILA, ELA, etc.) to use for a certain type of networkcontent, and how result(s) of technique(s) is to be processed.

Computer Architecture

As described previously, module 110 can be implemented using software,hardware, or combination therefore. However, those skilled in the artunderstand that a computer system may also be used to implement module110 to perform the functions described herein. FIG. 7 is a block diagramthat illustrates an embodiment of a computer system 700 upon whichembodiments of the method 200 may be implemented. Computer system 700includes a bus 702 or other communication mechanism for communicatinginformation, and a processor 704 coupled with bus 702 for processinginformation. Computer system 700 also includes a main memory 706, suchas a random access memory (RAM) or other dynamic storage device, coupledto bus 702 for storing information and instructions to be executed byprocessor 704. Main memory 706 also may be used for storing temporaryvariables or other intermediate information during execution ofinstructions to be executed by processor 704. Computer system 700 mayfurther include a read only memory (ROM) 708 or other static storagedevice coupled to bus 702 for storing static information andinstructions for processor 704. A data storage device 710, such as amagnetic disk or optical disk, is provided and coupled to bus 702 forstoring information and instructions.

Computer system 700 may be coupled via bus 702 to a display 712, such asa cathode ray tube (CRT), for displaying information to user 104. Aninput device 714, including alphanumeric and other keys, is coupled tobus 702 for communicating information and command selections toprocessor 704. Another type of user input device is cursor control 716,such as a mouse, a trackball, or cursor direction keys for communicatingdirection information and command selections to processor 704 and forcontrolling cursor movement on display 712. This input device typicallyhas two degrees of freedom in two axes, a first axis (e.g., x) and asecond axis (e.g., y), that allows the device to specify positions in aplane.

Computer system 700 can be used for processing network traffic content.According to some embodiments, such use may be provided by computersystem 700 in response to processor 704 executing one or more sequencesof one or more instructions contained in the main memory 706. Suchinstructions may be read into main memory 706 from anothercomputer-readable medium, such as storage device 710. Execution of thesequences of instructions contained in main memory 706 causes processor704 to perform the process steps described herein. One or moreprocessors in a multi-processing arrangement may also be employed toexecute the sequences of instructions contained in main memory 706. Inalternative embodiments, hard-wired circuitry may be used in place of orin combination with software instructions to implement embodimentsdescribed herein. Thus, embodiments described herein are not limited toany specific combination of hardware circuitry and software.

The term “computer-readable medium” as used herein refers to any mediumthat participates in providing instructions to processor 704 forexecution. Such a medium may take many forms, including but not limitedto, non-volatile media, and volatile media. Non-volatile media includes,for example, optical or magnetic disks, such as storage device 710.Volatile media includes dynamic memory, such as main memory 706.

Common forms of computer-readable media include, for example, a floppydisk, a flexible disk, hard disk, magnetic tape, or any other magneticmedium, a CD-ROM, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, a RAM, a PROM, and EPROM,a FLASH-EPROM, any other memory chip or cartridge, a carrier wave asdescribed hereinafter, or any other medium from which a computer canread.

Various forms of computer-readable media may be involved in carrying oneor more sequences of one or more instructions to processor 704 forexecution. For example, the instructions may initially be carried on amagnetic disk of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 700 canreceive the data on the telephone line and use an infrared transmitterto convert the data to an infrared signal. An infrared detector coupledto bus 702 can receive the data carried in the infrared signal and placethe data on bus 702. Bus 702 carries the data to main memory 706, fromwhich processor 704 retrieves and executes the instructions. Theinstructions received by main memory 706 may optionally be stored onstorage device 710 either before or after execution by processor 704.

Computer system 700 also includes a communication interface 718 coupledto bus 702. Communication interface 718 provides a two-way datacommunication coupling to a network link 720 that is connected to alocal network 722. For example, communication interface 718 may be anintegrated services digital network (ISDN) card or a modem to provide adata communication connection to a corresponding type of telephone line.As another example, communication interface 718 may be a local areanetwork (LAN) card to provide a data communication connection to acompatible LAN. Wireless links may also be implemented. In any suchimplementation, communication interface 718 sends and receiveselectrical, electromagnetic or optical signals that carry data streamsrepresenting various types of information.

Network link 720 typically provides data communication through one ormore networks to other devices. For example, network link 720 mayprovide a connection through local network 722 to a host computer 724.Network link 720 may also transmits data between an equipment 726 andcommunication interface 718. The data streams transported over thenetwork link 720 can comprise electrical, electromagnetic or opticalsignals. The signals through the various networks and the signals onnetwork link 720 and through communication interface 718, which carrydata to and from computer system 700, are exemplary forms of carrierwaves transporting the information. Computer system 700 can sendmessages and receive data, including program code, through thenetwork(s), network link 720, and communication interface 718. Althoughone network link 720 is shown, in alternative embodiments, communicationinterface 718 can provide coupling to a plurality of network links, eachof which connected to one or more local networks. In some embodiments,computer system 700 may receive data from one network, and transmit thedata to another network. Computer system 700 may process and/or modifythe data before transmitting it to another network.

Although particular embodiments have been shown and described, it willbe understood that it is not intended to limit the present inventions tothe preferred embodiments, and it will be obvious to those skilled inthe art that various changes and modifications may be made withoutdeparting from the spirit and scope of the present inventions. Forexample, in other embodiments, one or more functions performed by module110 may be implemented using one or more processors or one or moresoftware. Also, in alternative embodiments, module 110 needs not performall of the steps in FIG. 2. For example, in other embodiments, module110 does not analyze content data (i.e., does not perform Step 202), butreceives results of analysis from a source. In such cases, based onresults of analysis received, module 110 then categorize network trafficcontent based on accuracies of the results. The specification anddrawings are, accordingly, to be regarded in an illustrative rather thanrestrictive sense. The present inventions are intended to coveralternatives, modifications, and equivalents, which may be includedwithin the spirit and scope of the present inventions as defined by theclaims.

What is claimed is:
 1. In a content control server coupled to senderdevices and a user device over a data communication network, acomputer-implemented method for improving content blocking based oncategorizing network traffic content from network traffic content sentbetween the sender device and the user device, the method comprising thesteps of: receiving, via a network interface device of a networkeddevice on which the method is implemented through instructionsexecutable by at least one processor, the network traffic contentincluding an electronic message; determining, through execution ofinstructions of an electronic message content categorization module onthe at least one processor, a first characterization of the networktraffic content, the first characterization including that theelectronic message likely includes undesirable content, the determiningperformed according to at least one analysis technique to obtain atleast one categorization of the electronic message indicating theelectronic message likely includes undesirable content, the at least oneanalysis performed as a function of a database of known categorizationproperties of electronic message content; determining whether the firstcharacterization meets a plurality of prescribed weighted accuracythresholds, including: collecting results for a technique; determining aprobability curve based on a statistical analysis of results of atechnique, each weight values corresponding to a probability dependingupon accuracy of the results; assigning the prescribed weighted accuracythresholds with a specific weight value to modify the firstcharacterization by applying the specific weight value to the firstcharacterization in accordance with the first characterization fallingwithin the prescribed weighted accuracy threshold, further wherein theprescribed weighted accuracy thresholds include an exclusionaryprescribed weighted accuracy threshold; responsive to a determinationthat the first characterization meets one of the plurality of prescribedweighted accuracy thresholds, other than the exclusionary prescribedweighted accuracy threshold: creating a weighted first characterizationwherein the weighted first characterization applies the specific weightvalue to modify the first characterization in accordance with the metone prescribed weighted accuracy threshold; categorizing the electronicmessage content based at least in part on the weighted firstcharacterization; storing, on a data storage device, by the electronicmessage content categorization module, a representation of thedetermination that the electronic message likely includes undesirablecontent and data derived from the electronic message such that thestored representation is available to assist in processing subsequentlyreceived electronic messages included in network traffic content;responsive to a determination that the first characterization meets theexclusionary prescribed weighted accuracy threshold, dismissing thefirst characterization; and blocking future network content based on thefirst characterization.
 2. The method of claim 1, wherein storing therepresentation of the determination that the electronic message likelyincluded undesirable content and data derived from the electronicmessage includes storing the representation in the database of knowncategorization properties of electronic message content.
 3. The methodof claim 1, wherein the electronic message comprises text-based chatmessages.
 4. The method of claim 1, wherein the electronic message is anemail.
 5. The method of claim 1, wherein the network traffic contentcomprises content received over a network according to the HypertextTransport Protocol and an email protocol.
 6. The method of claim 1,wherein the at least one analysis technique includes at least one of aninternal link analysis, an external link analysis, a meta tag analysis,and a token analysis.
 7. The method of claim 1, wherein the electronicmessage content categorization module receives a copy of the electronicmessage content to build the database of known categorization propertiesof electronic message content and an electronic message contentscreening module screens electronic message content destined for usersaccording to the database of known categorization properties ofelectronic message content.
 8. A non-transitory device-readable storagemedium, with instructions thereon which when executed by at least oneprocessor of a network device, causes the network device to screenelectronic message content by: receiving, via a network interface deviceof a networked device on which the method is implemented throughinstructions executable by at least one processor, the network trafficcontent including an electronic message; determining, through executionof instructions of an electronic message content categorization moduleon the at least one processor, a first characterization of the networktraffic content, the first characterization including that theelectronic message likely includes undesirable content, the determiningperformed according to at least one analysis technique to obtain atleast one categorization of the electronic message indicating theelectronic message likely includes undesirable content, the at least oneanalysis performed as a function of a database of known categorizationproperties of electronic message content; determining whether the firstcharacterization meets a plurality of prescribed weighted accuracythresholds, including: collecting results for a technique; determining aprobability curve based on a statistical analysis of results of atechnique, each weight values corresponding to a probability dependingupon accuracy of the results; assigning the prescribed weighted accuracythresholds with a specific weight value to modify the firstcharacterization by applying the specific weight value to the firstcharacterization in accordance with the first characterization fallingwithin the prescribed weighted accuracy threshold, further wherein theprescribed weighted accuracy thresholds include an exclusionaryprescribed weighted accuracy threshold; responsive to a determinationthat the first characterization meets one of the plurality of prescribedweighted accuracy thresholds, other than the exclusionary prescribedweighted accuracy threshold: creating a weighted first characterizationwherein the weighted first characterization applies the specific weightvalue to modify the first characterization in accordance with the metone prescribed weighted accuracy threshold; categorizing the electronicmessage content based at least in part on the weighted firstcharacterization; storing, on a data storage device, by the electronicmessage content categorization module, a representation of thedetermination that the electronic message likely includes undesirablecontent and data derived from the electronic message such that thestored representation is available to assist in processing subsequentlyreceived electronic messages included in network traffic content;responsive to a determination that the first characterization meets theexclusionary prescribed weighted accuracy threshold, dismissing thefirst characterization; and blocking future network content based on thefirst characterization.
 9. The non-transitory device-readable storagemedium of claim 8, wherein storing the representation of thedetermination that the electronic message likely included undesirablecontent and data derived from the electronic message includes storingthe representation in the database of known categorization properties ofelectronic message content.
 10. The non-transitory device-readablestorage medium of claim 8, wherein the electronic message comprisestext-based chat messages.
 11. The non-transitory device-readable storagemedium of claim 8, wherein the electronic message is an email.
 12. Thenon-transitory device-readable storage medium of claim 8, wherein thenetwork traffic content comprises content received over a networkaccording to the Hypertext Transport Protocol and an email protocol. 13.The non-transitory device-readable storage medium of claim 8, whereinthe at least one analysis technique includes at least one of an internallink analysis, an external link analysis, a meta tag analysis, and atoken analysis.
 14. The non-transitory device-readable storage medium ofclaim 8, wherein the electronic message content categorization modulereceives a copy of the electronic message content to build the databaseof known categorization properties of electronic message content and anelectronic message content screening module screens electronic messagecontent destined for users according to the database of knowncategorization properties of electronic message content.
 15. A contentcontrol server coupled to sender devices and a user device over a datacommunication network, the content control server for improving contentblocking based on categorizing network traffic content from networktraffic content sent between the sender device and the user device, thecontent control server comprising: at least one processor; at least onememory device; at least one network interface device; an electronicmessage content categorization module stored on the at least one memorydevice and executable by the at least one processor to: receive, via anetwork interface device of a networked device on which the method isimplemented through instructions executable by at least one processor,network traffic content including an electronic message; determine,through execution of instructions of an electronic message contentcategorization module on the at least one processor, a firstcharacterization of the network traffic content, the firstcharacterization including that the electronic message likely includesundesirable content, the determining performed according to at least oneanalysis technique to obtain at least one categorization of theelectronic message indicating the electronic message likely includesundesirable content, the at least one analysis performed as a functionof a database of known categorization properties of electronic messagecontent; determine whether the first characterization meets a pluralityof prescribed weighted accuracy thresholds, including: collectingresults for a technique; determining a probability curve based on astatistical analysis of results of a technique, each weight valuescorresponding to a probability depending upon accuracy of the results;assigning the prescribed weighted accuracy thresholds with a specificweight value to modify the first characterization by applying thespecific weight value to the first characterization in accordance withthe first characterization falling within the prescribed weightedaccuracy threshold, further wherein the prescribed weighted accuracythresholds include an exclusionary prescribed weighted accuracythreshold; responsive to a determination that the first characterizationmeets one of the plurality of prescribed weighted accuracy thresholds,other than the exclusionary prescribed weighted accuracy threshold:create a weighted first characterization wherein the weighted firstcharacterization applies the specific weight value to modify the firstcharacterization in accordance with the met one prescribed weightedaccuracy threshold; store, on a data storage device, by the electronicmessage content categorization module, a representation of thedetermination that the electronic message likely includes undesirablecontent and data derived from the electronic message such that thestored representation is available to assist in processing subsequentlyreceived electronic messages included in network traffic content;responsive to a determination that the first characterization meets theexclusionary prescribed weighted accuracy threshold, dismiss the firstcharacterization; and block future network content based on the firstcharacterization.
 16. The network device of claim 15, wherein storingthe representation of the determination that the electronic messagelikely included undesirable content and data derived from the electronicmessage includes storing the representation in the database of knowncategorization properties of electronic message content.
 17. The networkdevice of claim 15, wherein the electronic message comprises text-basedchat messages.
 18. The network device of claim 15, wherein theelectronic message is an email.
 19. The network device of claim 15,wherein the network traffic content comprises content received over anetwork according to the Hypertext Transport Protocol and an emailprotocol.
 20. The network device of claim 15, wherein the at least oneanalysis technique includes at least one of an internal link analysis,an external link analysis, a meta tag analysis, and a token analysis.